Color by Numbers - Quantifying the Quality of Color Reproduction

Quantifying the perceptual difference between original and reproduced (and inevitably modified) color images is currently a key research challenge in the field of color imaging. Such information can be extremely valuable for instance in the development of new equipment and algorithms for color reproduction. While in many research areas it is common practice to obtain quantitative quality information by the use of perceptual tests, in which the judgments of several human observers are being collected and carefully analyzed statistically, this approach has serious limitations for practical use, in particular because of the time consumption. Motivated by this, and aided by the ever increasing available knowledge about the mechanisms of the human visual system, the quest for perceptual color image quality metrics that can adequately predict human quality judgments of complex images, has been on for several decades. However, unfortunately, the Holy Grail is yet to be found. The current paper outlines the state of the art of this field, including benchmarking of existing metrics, presents recent research, and proposes promising areas for further work. Aspects that are covered in particular include new models and metrics for color image quality, and new frameworks for using the metrics to improve color image representation and reproduction algorithms. Introduction Many of the unsolved research problems in color reproduction of digital images today can be somewhat loosely described as trying to achieve certain goals without destroying the images too much. For instance for image compression the goal is typically to reduce the storage requirements (i.e. file size) as much as possible while keeping the image as true as possible to the uncompressed original, i.e. without introducing visible artifacts. For cross-media color reproduction, one is typically faced with the challenge of reproducing images on different devices that are not physically capable of reproducing all the colors contained in the images; research on color gamut mapping attempts to find algorithms which consistently result in most accurate and pleasing images despite the device limitations. For image reproduction scenarios where exact reproduction of original images or documents is the goal (e.g. colour facsimile or artwork reproduction), the optimal image quality would be achieved when there is as little as possible perceived visual difference between the original and the reproduction. Measuring the perceived difference between color images is thus a fundamental step for representation and reproduction of digital images. In different fields different approaches have been taken. For instance in image compression research it has been common practice to use the mean square error (MSE) pixel difference between the original and compressed images as a quality indicator, while for color gamut mapping, perceptual tests involving visual inspection of images by a number of observers are typically performed. The former approach is computationally simple but has been proven to correlate poorly with the perceived image difference; the latter is perceptual in its nature but is prohibitively time-consuming, subjective, and non quantitative. Similar approaches are being used in other color imaging research fields, such as color quantization, halftoning, image sensor improvement, restoration, demosaicing, and visualization of high dynamic range imagery. In recent years, several attempts have been made to develop image difference metrics that correlate better with the perceived difference, but there is still a huge knowledge gap on this topic. We are here faced with the difficult problem of trying to find a numerical algorithm based on an analysis of the pixels of two digital images, which correlates with the visual perception of the difference between these same images. In particular it is often the case that new metrics show relatively satisfactory performance for specific applications, but fail severely to generalize to other applications. Recent research in our research laboratory has attempted to fill this gap by developing novel generic and accurate perceptual image difference metrics, as well as by developing and evaluating a framework for improving image representation and reproduction algorithms by using such metrics. The current presentation will present selected aspects of this recent research [1-9]. Selected recent research As already mentioned, a large number of image quality metrics has been proposed in the literature. In our recent survey [4], we described more than 100 different full-reference image quality metrics. But still, as demonstrated for instance on gamut mapped images [6] and by the recent extensive tests of Ajagamelle et al [9] on different databases with varying image distortions; the metrics do not perform satisfactorily for complex and general distortions. We have, however achieved competitive results with some new approaches to quantifying image difference, by combining contrast sensitivity function based filtering with hue angle weighting [5] and by using bilateral filtering to preserve the perceptually important edges in the images [1]. On the other hand, for specific distortions which occur in specific applications or color reproduction workflows, it is indeed possible to achieve useful predictions of image quality from today’s metrics. A key aspect in this regards is to consider image quality not as a mono-dimensional entity, but as being constituted of a series of Quality Attributes (QA). In [2] we have identified and discussed the most important QAs when evaluating the quality of prints, they are as follows: Color contains aspects related to the perceived color, such as hue, saturation, and color rendition, except lightness. Lightness is considered so perceptually important that it is beneficial to separate it from the color QA. Lightness will range from “light” to “dark”. CGIV 2010 Final Program and Proceedings 429 Contrast can be described as the perceived magnitude of visual meaningful differences, global and local, in lightness and chromaticity within the image. Sharpness is related to the clarity of details and definition of edges. In color printing some artifacts can be perceived in the resulting image. These artifacts, like noise, contouring, and banding, contribute to degrading the quality of an image if detectable. The physical QA contains all physical parameters that affect quality, such as paper and gloss. Currently, work is underway to validate these QAs and to propose metrics which can estimate them. When considering color reproduction on print, new challenges arise for quantifying the resulting quality. In [8], a framework for applying image quality metrics to printed images is proposed, including the transformation to a digital format, image registration, and the application of image quality metrics. The proposed framework introduces less error and is significantly faster than another state of the art framework [10]. The framework is also used to evaluate a set of image quality metrics against subjective data. Summary This paper has only given a very brief taste of the issues involved in quantifying the quality of color reproduction Acknowledgments I am extremely grateful to my co-workers at the Colorlab who have contributed so extensively to this research. Many of them would not have been with me if it had not been for the funding provided by the Research Council of Norway and by Océ Technologies B.V, for this I sincerely thank you.